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AN  EVALUATION  OF  THE  PERFORMANCE  OF  A 
NEW  STORM  TRACKING  METHODOLOGY 


ABSTRACT 


This  report  contains  the  results  of  an  exploratory 
statistical  analysis  to  evaluate  the  performance 
of  the  Systematic  Error  Identification  System  (SEIS) 
and  the  Vortex  Tracking  Program  (VTP) ,  when 
tracking  weather  systems. 


AN  EVALUATION  OF  THE  PERFORMANCE  OF  A 
NEW  STORM  TRACKING  METHODOLOGY 

1 .   Introduction 

Weather  forecasts  made  by  the  Fleet  Numerical  Oceanography 

Center  (FNOC)  are  based  on  a  numerical  weather  prediction  model 

called  the  Naval  Operational  Global  Atmospheric  Prediction 

System  (NOGAPS) .   Until  1983  the  only  available  measures  of 

model  performance  were  of  a  global  nature  (aggregated  over 

all  the  weather  systems  monitored),  such  as  means,  variances 

and  root  mean  square  errors.   The  operational  field  forecasters, 

on  the  other  hand,  prefer  error  statistics  relevant  at  the 

synoptic  level,  i.e.,  measures  pertaining  to  forecasts  of 

individual  storms  and  troughs.   Such  measures  would  enable 

these  forecasters  to  provide  better  subjective  forecasts  at 

the  regional  level.   In  1982,  the  Naval  Environmental  Prediction 

Research  Facility  (NEPRF)  began  the  development  of  the  Systematic 

Error  Identification  System  (SEIS) ;  the  primary  data  reduction 

methodology  within  SEIS  is  the  Vortex  Tracking  Program  (VTP) . 

In  the  VTP,  an  atmospheric  low/high  pressure  system  is 

mathematically  represented  as  a  generalized  six  parameter 

elliptic  function.   The  six  parameters  correspond  with  the 

primary  features  of  a  storm,  viz.,  the  amplitude  A,  R  the 

semi-major  of  the  elliptic  representation  of  the  storm,  z    the 

eccentricity  or  the  ratio  of  the  semi-major  to  the  semi-minor, 

a  the  orientation  of  the  ellipse  and   X  ,  Y    the  coordinates 

c  o    o 

of  the  center  of  the  storm.   The  units  of  measurement  are 

millibars  (mb)  for  A,  and  degrees  with  respect  to  the  North 

for  a  while  R,  X  ,  Y    are  measured  in  terms  of  a  63x63  FNOC 

o    o 

hemispheric  grid  units.   For  each  valid  storm,  the  VTP 

uses  an  iterated  non-linear  least  squares  scheme  to  estimate 

A,  R,  e,  a,  X  ,  Y   within  the  sea  level  pressure  field  for 

the  analysis  at  time  t  as  well  as  for  the  associated 

12,  24,  36,  48  and  60  hour  forecasts  produced  by  NOGAPS.   The 

iteration  scheme  requires  a  set  of  initial  guess  values 

for  the  parameters  to  produce  the  estimates  for  the  analysis 

field  at  time  t.   These  estimates  are  in  turn  used  as  guess 

values  to  produce  the  12  hour  forecasts;  the  12  hour  parameter 
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forecasts  are  used  to  generate  the  24  hour  forecasts  and  so 
on.   The  estimated  parameter  values  for  the  analysis  field 
at  time  t  are  also  used  as  the  first  guesses  for  the  analysis 
field  at  t  +  12  hours.   The  estimates  for  the  analysis  field 
are  usually  referred  to  as  verification  values.   Corresponding 
to  each  set  of  forecasted  parameter  values  there  will  be  a 
verification  set  obtained  using  the  current  (for  the  forecasted 
time)  sea  level  pressure  data.   The  difference  between  a 
forecasted  value  and  its  verification  value  is  called  the 
forecast  error.   SEIS,  thus,  provides  the  capability  to  track 
individual  weather  systems  (by  tracking  the  movement  of  the 
elliptic  representation)  and  also  a  means  to  measure  and 
analyze  the  tracking  errors. 

The  modified  NOGAPS  model  has  been  running  on  a  real 
time  basis  since  mid  1983.   During  the  life  cycle  of  each 
valid  storm,  twice  each  day  (at  noon  and  at  midnight  GMT) , 
the  elliptic  parameter  estimates  are  produced  for  the  analysis 
field  and  the  associated  12,  24,  36,  48  and  60  hour  forecast 
fields.   References  [1],  [2]  and  [3]  discuss  the  VTP  and  SEIS 
models  in  more  detail. 

The  objective  of  this  project  is  the  exploratory 
statistical  analysis  of  the  forecast  errors  to  assess  the 
performance  of  the  SEIS/VTP  model.   Data  on  80  storms, 
covering  the  North  Pacific  Ocean  Basin,  observed  during  the 
period  January-May  1984  has  been  used  in  this  study.   The 
results  of  the  analysis  are  described  in  the  following 
sections.   Section  2  contains  overall  measures  of  performance 
of  SEIS/VTP,  primarily  summary  statistics  of  forecast  errors 
pooled  over  all  the  80  storms.   Error  statistics  pertaining 
to  the  tracking  of  individual  storms  are  presented  in  Section  3 
Conclusions  and  topics  for  further  research  are  discussed  in 
Section  4 . 
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2.   Analysis  of  Forecast  Errors 

A  forecast  error  is  defined  as  the  difference  between 
a  forecasted  parameter  value  and  its  verification  value; 
an  absolute  forecast  error  is  the  absolute  value  of  a 
forecast  error.   For  each  of  the  five  forecasting  periods 
(12,  24,  36,  48  and  60  hours)  the  forecast  and  the  absolute 
forecast  errors  were  subjected  to  various  statistical 
analyses.   Tables  1  and  2  contain  the  means  (X)  and  standard 
deviations   (S)   for  these  errors. 


TABLE  1 
SUMMARY  STATISTICS  OF  FORECAST  ERRORS 


Forecast 
Period 


Number 
of 
Samples 


x 


12 
24 
36 
48 

60 


487 
429 
371 
329 
288 


1.53 
2.67 

•4.11 
•5.14 
•5.08 


5.20 
6.69 
7.68 
8.50 
9.61 


■0.05 
•0.07 
•0.13 
•0.18 
•0.30 


1.49 
1.47 
1.86 
1.84 
2.03 


0.21 
0.19 
0.27 
0.28 
0.21 


1.27 
1.38 
1.73 
1.76 
1.85 


4.75 
4.60 
4.83 
5.51 
5.10 


49.83 
52.14 
56.10 
53.95 
50.85 


•0.06 
•0.08 
•0.14 
■0.11 
■0.09 


0.77 
1.01 
1.22 
1.34 
1.45 


0.1 
0.1 
-0. 
-0. 
-0. 


TABLE  2 
SUMMARY  STATISTICS  OF  ABSOLUTE  FORECAST  ERRORS 


Forecast 
Period 

12 

24 

36 

48 

60 


Number 
of 
Samples 

487 

429 

371 

329 

288 


3.87 

5.56 
6.84 
7.65 
8.  38 


3.79 
4.57 
5.39 
6.32 
6.92 


x 


0.85 
0.92 
1.07 
1.07 
1.20 


1.23 
1.15 
1.52 
1.51 
1.67 


0.85 
1.00 
1.19 
1.19 
1.20 


0.98 
0.98 
1.28 
1.32 
1.42 


28.79 
32.65 
37.26 
36.65 
35.20 


40.93 
40.89 
42.18 
39.92 
36.99 


X 


0.50 
0.69 
0.87 
0.97 
1.04 


0.59 
0.73 
0.86 
0.93 
1.01 


0.5 

0.7 

l.i 

1.2 

1.3 
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The  following  general  conclusions  appear  warranted. 
The  NOGAPS  forecasting  methodology  does  forecast  the  parameters 
A,  R,  e,  X  ,    Y    quite  well.   With  regards  to  the  forecasting 
of  the  orientation  a,  although  the  mean  errors  are  not 
excessive,  the  standard  deviations  are  somewhat  high.   In 
many  cases  the  forecast  errors  are  negative  indicating  a 
negative  bias,  i.e.,  the  forecasted  values  tend  to  be  on 
the  low  side  of  the  verification  values.   With  a  few 
exceptions,  the  means  and  standard  deviations  increase  with 
an  increase  in  the  forecasting  period;  this  is  to  be  expected 
in  view  of  the  higher  levels  of  uncertainty  involved. 
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The  autocorrelations  for  lags  1  to  5  between  the 

forecast  errors  are  presented  in  Table  3.   Except  for 

the  lag-one  autocorrelations  of  about  .30  for  the  forecast 

errors  of  A,  X  ,  Y   the  rest  of  the  autocorrelations 
o   o 

are  quite  negligible.   This  implies  that  a  large  error  in 
forecasting  a  parameter  at  a  given  time  will  not  have  a 
lasting  effect  on  future  forecast  errors.   Also,  the 
correlation  matrices  (correlations  between  the  errors  in 
forecasting  A,  a ,  R,  e ,  X  ,  Y  )  in  Table  4  show  that  these 
correlations  are  negligible  with  one  exception  —  the 
correlation  between  the  errors  in  forecasting  R  and  e 
is  around  .5.   This  may  be  interpreted  to  mean  (with  the 
one  exception)  that  a  large  forecast  error  for  one  parameter 
will  not  have  a  detrimental  effect  on  the  estimates  of  the 
other  parameters. 

In  an  attempt  to  model  the  statistical  behavior, 
gamma  distribution  were  fit  to  the  12,  24,  36  and  48  hour 
absolute  forecast  errors.   The  histograms  with  the  fitted 
gamma  distribution  superimposed  are  in  Figures  1-18.   Gamma 
distributions  appear  to  serve  as  good  statistical  models 
of  the  absolute  forecast  errors  for  A  and  R.   In  the  other 
cases,  the  lack  of  fit  may  be  attributed  to  a  higher  peakedness 
in  the  data;  a  Weibull  distribution  may  provide  a  better  fit. 
Although  no  graphs  are  presented,  gamma  distributions  did  not 
provide  a  good  fit  to  the  forecast  errors  (appropriately 
translated/shifted  to  make  them  positive)  also.   Further 
work  will  be  necessary  to  determine  the  most  appropriate 
statistical  distributions  to  model  the  probabilistic  behavior 
of  the  forecast  errors.   Proper  statistical  modeling  of  the 
error  data  could  be  useful  for  exploring  the  development  of 
uncertainty  contours  (confidence  regions)  for  the  movements 
of  weather  systems. 
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3 .   Statistics  of  Individual  Weather  Systems 

To  evaluate  the  performances  of  SEIS,  in  tracking 
individual  weather  systems,  and  the  NOGAPS  model  in 
forecasting  weather  systems,  data  on  20  storms  with  at 
least  10  records  per  storm  (i.e.,  10  sets  of  forecasted  and 
verification  values  per  storm)  were  examined.   The  means   (X) 
and  standard  deviations   (S)   of  the  forecast  errors  for  these 
20  storms  are  in  Table  5. 

The  trends  in  the  forecast  errors  are  similar  to  the 
global  trends  observed  in  the  previous  section;  with  the 
exception  of  the  forecasting  of  a,  the  forecast  errors  are 
very  small  even  at  the  individual  storm  level.   The  iterated 
non-linear  least  square  procedure  in  VTP  requires  initial 
guess  values  for  each  of  the  parameters  A,  e,    R,  a,  X  ,  Y  ; 
the  initial  guess  for  a  is  always  specified  as  zero.   We 
conjecture  that  this  may  be  the  cause  of  the  somewhat  erratic 
forecasts  of  a.   A  better  initial  guess,  closer  to  the  true 
value,  may  result  in  a  better  forecast  of  a.   The  SEIS/VTP 
appears  to  be  exceptionally  good  in  forecasting  the  center 
of  a  storm. 

For  each  of  the  20  storms  the  forecasted  values  of 
A,  X   and  Y   were  plotted  against  their  respective  verification 
values.   In  several  cases,  the  scatter  plots  indicated  an 
approximate  linear  relationship  between  the  forecasted  and 
verification  values.   A  few  of  these  scatter  plots  are 
shown  in  Figures  19-30.   A  linear  regression  analysis  was, 
therefore,  performed  with  the  forecasted  value  as  the 
independent  variable  and  the  verification  value  as  the 
dependent  variable.   The  least  squares  estimates  of  the 
intercept  and  slope  of  the  fitted  line  and  also  the  estimated 
coefficient  of  correlation  (a  measure  of  goodness  of  the 
fitted  line)  are  in  Table  6. 
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SUMMARY  STATISTICS  OF 


TABLE  5 
FORECAST  ERRORS  FOR  INDIVIDUAL  STORMS 


' 

I ■ 

:>rm 

Number 

of 
Records 

A 

e 

R 

a 

X 
0 

X 

0 

nber 

X 

s 

X 

s 

X 

s 

X 

s 

X 

s 

X 

s 

i 

19 

-9.9 

5.2 

0.6 

1.3 

0.  8 

1.5 

4.4 

99.0 

5.3 

0.8 

0.1 

0.3 

2 

18 

-10.3 

5.1 

0.5 

1.2 

0.7 

1.5 

4.0 

101.7 

0.1 

0.8 

-0.1 

0.3 

3 

34 

-1.2 

4.9 

-0.6 

1.5 

-0.  3 

1.9 

9.3 

24.5 

-5.2 

0.7 

-1.1 

1.4 

4 

18 

-6.4 

5.1 

-0.7 

2.8 

0.5 

2.8 

-3.3 

20.3 

i.i 

1.4 

-0.8 

1.3 

5 

60 

-5.6 

10.1 

0.1 

0.9 

0.6 

1.4 

-5.0 

35.5 

0.4 

1.1 

-0.5 

1.8 

1 

19 

-1.8 

3.9 

-0.2 

1.0 

1.4 

1.7 

6.9 

31.3 

-1.7 

1.3 

3.2 

1.8 

1 

27 

1.3 

3.9 

0.1 

0.4 

0.5 

0.9 

34.9 

32.5 

0.1 

0.8 

-0.9 

1.4 

8 

41 

-8.0 

9.3 

-1.3 

3.3 

0.7 

2.3 

3.5 

47.2 

-0.7 

1.3 

0.7 

1.4 

9 

25 

-3.2 

16.0 

-2.6 

4.0 

-0.5 

1.4 

11.0 

74.  8 

-1.1 

1.6 

0.5 

0.  8 

0 

26 

4.8 

7.2 

-0.1 

2.4 

0.4 

1.1 

-13.4 

86.2 

0.8 

1.6 

-0.4 

0.5 

1 

23 

-6.5 

12.8 

0.4 

0.6 

0.7 

1.6 

9.3 

55.5 

0.1 

1.3 

0.0 

0.5 

2 

13 

0.4 

5.5 

1.5 

2.3 

0.2 

0.  7 

-10.7 

29.2 

0.1 

0.9 

0.0 

0.5 

3 

63 

-2.4 

8.3 

-0.1 

1.7 

-0.3 

1.1 

-6.9 

54.4 

0.5 

1.1 

0.8 

0.6 

4 

10 

-11.6 

9.6 

-0.1 

0.8 

-0.8 

1.1 

22.1 

66.6 

1.0 

1.0 

-0.2 

0.5 

5 

20 

-0.7 

6.8 

0.9 

1.4 

0.9 

2.1 

-7.3 

36.  7 

1.2 

2.0 

-0.9 

2.0 

6 

27 

-2.0 

6.7 

-1.1 

1.2 

-0.9 

2.0 

7.5 

33.0 

-0.1 

1.3 

0.1 

1.2 

7 

13 

-6.6 

4.2 

-0.  3 

0.7 

-0.7 

0.8 

-17.6 

17.2 

0.0 

0.5 

0.3 

0.5 

8 

95 

-3.4 

6.7 

-0.2 

1.1 

0.2 

1.5 

12.9 

52.0 

0.2 

1.1 

0.9 

2.2 

9 

72 

-5.1 

9.4 

0.  3 

1.4 

-0.1 

1.1 

-11.0 

54.8 

0.1 

1.0 

0.1 

1.9 

0 

51 

-3.8 

6.5 

-0.5 

2.7 

-1.3 

2.9 

-20.1 

36.8 

-0.4 

0.5 

0.7 

0.7 
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TABLE  6 
ESTIMATED  REGRESSION  PARAMETERS 


Storm 

A 

X 

0 

Y 

0 

No. 

Intercept 

Slope 

Correlation 

Intercept 

Slope  Correlation 

Intercept 

Slope  ( 

3orrelat3 

1 

212.9 

0.79 

0.91 

12.1 

0.45 

0.55 

28.8 

0.25 

0.51 

2 

630.2 

0.37 

0.90 

15.5 

0.25 

0.64 

38.9 

-0.05 

0.45 

3 

984.9 

0.003 

0.00 

2.7 

0.84 

0.92 

22.9 

0.29 

0.3C 

4 

495.  8 

0.50 

0.37 

20.9 

0.09 

0.16 

23.9 

0.09 

0.J 

5 

732.7 

0.26 

0.  33 

11.3 

0.35 

0.70 

13.1 

0.62 

o.8; 

6 

962.7 

0.04 

0.05 

11.3 

0.45 

0.14 

-7.3 

1.12 

0.7 

7 

589.3 

0.41 

0.75 

2.0 

0.88 

0.75 

15.5 

0.61 

0.9 

8 

357.  8 

0.64 

0.64 

7.5 

0.68 

0.87 

12.4 

0.57 

0.8 

9 

1066.7 

-0.08 

0.09 

8.5 

0.63 

0.85 

8.7 

0.69 

0.9 

10 

714.  8 

0.28 

0.28 

-0.7 

0.99 

0.82 

15.1 

0.46 

0.4 

11 

626.6 

0.37 

0.29 

25.9 

-0.20 

-0.29 

4.6 

0.88 

0.8 

12 

143.1 

0.86 

0.47 

6.1 

0.73 

0.80 

6.8 

0.75 

0.8 

13 

745.0 

0.25 

0.47 

2.0 

0.87 

0.86 

-0.2 

0.98 

0.9 

14 

-133.0 

1.14 

0.  33 

-4.0 

1.19 

0.48 

10.7 

0.59 

0.8 

15 

446.2 

0.55 

0.67 

7.9 

0.60 

-0.03 

8.8 

0.75 

-0.2 

16 

640.4 

0.  35 

0.79 

-1.6 

1.07 

0.80 

-0.6 

1.01 

0.6 

17 

349.5 

0.65 

0.09 

4.1 

0.81 

0.32 

0.4 

0.97 

0.3 

18 

812.9 

0.18 

0.59 

8.4 

0.52 

0.60 

0.3 

0.99 

0.8 

19 

261.3 

0.74 

0.57 

22.5 

-0.02 

0.95 

30.1 

-0.09 

0.9 

20 

959.5 

0.04 

0.81 

10.1 

0.50 

0.91 

27.9 

0.24 

0.9 
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The  regression  analyses  confirmed  what  was  observed 
from  the  scatter  plots,  namely,  a  linear  fit  in  many  cases. 
If  the  functional  relationship  between  the  forecasted  and 
verification  values  can  be  determined  with  good  precision, 
corrective  action  can  be  taken  to  remove  this  source  of 
systematic  error  in  the  forecasting  scheme.   However,  the 
results  in  Table  6  do  not  lend  themselves  to  the  determination 
of  the  functional  relationship.   More  data  needs  to  be 
examined  and  different  ways  of  stratifying  the  data  such  as 
by  geographic  regions  and/or  climatic  seasons  may  prove  to 
be  profitable.   Another  possibility  is  to  group  the  available 
records  for  a  storm  according  to  the  forecast  periods,  i.e., 
all  12  hour  forecasts  as  one  group,  all  24  hour  forecasts  as 
another  group,  etc.   Of  course,  this  scheme  can  only  be 
applied  to  storms  with  large  numbers  of  records.   We  tried 
this  approach  on  4  out  of  the  20  storms  (storms  5,  13,  18 
and  19,  each  with  60  or  more  records).   The  records  for 
each  storm  were  formed  into  five  groups,  one  for  each  of 
the  forecast  periods  and  a  separate  regression  analysis  was 
performed  for  each  group. 

The  estimated  regression  parameters  in  Table  7  reveal 

a  much  stronger  linear  relationship  when  the  data  is 

stratified  according  to  the  forecast  period.   Also,  one 

can  discern  a  definite  pattern  in  the  relationship  between 

the  forecasted  and  verification  values  of  the  storm's 

amplitude  A.   For  the  12  hour  forecasts,  the  relationship 

is  linear  with  an  intercept  value  of  about  260  and  slope  . 7 ; 

the  intercept  and  slope  values  for  the  48  hour  forecasts 

are  around  475  and  .5  respectively.   This  is  only  an 

empirical  observation  and  a  more  extensive  study  will  be 

necessary  to  confirm  this.   Even  though  there  is  a  strong 

correlation  between  the  forecasted  and  verification  values 

of  X   and  Y  ,  no  pattern  is  evident  in  the  estimates  of  the 
o       o      c 

intercepts  and  slopes  of  the  fitted  lines.   Another  observation 
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that  can  be  made  from  the  correlations  in  Table  7  is  that 
the  12  and  24  hour  forecasts,  and  to  a  lesser  extent  the 
36  hour  forecasts  correspond  well  with  the  verification 
values;  the  efficiency  of  the  forecasting  scheme  appears  to 
drop  after  the  36  hour  forecasts. 
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TABLE  7 
ESTIMATED  REGRESSION  PARAMETERS  FOR  INDIVIDUAL  FORECAST  PERIOD 


n  1 

Forecast 

A 

X 

0 

Yo 

Period     Intercept  Slope  Correlation 

Intercept  Slope  Correlation 

Intercept  Slope  ( 

Zorrelatio) 

i 

12 

275.3 

0.72 

0.77 

3.4 

0.81 

0.80 

10.6 

0.68 

0.83 

24 

476.2 

0.52 

0.72 

7.4 

0.56 

0.78 

6.0 

0.83 

0.96 

36 

748.6 

0.24 

0.37 

10.1 

0.40 

0.82 

12.6 

0.64 

0.85 

48 

1062.5    - 

-0.08 

-0.09 

12.7 

0.26 

0.66 

19.0 

0.45 

0.50 

60 

1235.6    ■ 

-0.25 

-0.24 

13.4 

0.23 

0.72 

22.5 

0.34 

0.61 

12 

454.6 

0.54 

0.55 

-2.0 

1.11 

0.96 

-0.1 

0.99 

0.99 

24 

572.5 

0.43 

0.52 

3.3 

0.81 

0.94 

-0.1 

0.98 

0.99 

36 

740.7 

0.26 

0.39 

6.4 

0.63 

0.78 

0.1 

0.97 

0.99 

48 

751.3 

0.25 

0.54 

4.4 

0.78 

0.80 

-1.4 

1.01 

0.99 

60 

762.9 

0.27 

0.74 

-1.0 

0.99 

0.78 

-1.0 

0.99 

0.99 

12 

256.9 

0.74 

0.77 

2.2 

0.90 

0.92 

0.1 

1.00 

0.96 

24 

489.0 

0.50 

0.49 

3.7 

0.81 

0.81 

2.8 

0.93 

0.93 

36 

454.4 

0.54 

0.60 

6.4 

0.67 

0.60 

8.1 

0.78 

0.81 

48 

616.6 

0.37 

0.45 

14.9 

0.25 

0.22 

14.8 

0.57 

0.66 

60 

437.2 

0.56 

0.61 

14.3 

0.27 

0.28 

16.  8 

0.51 

0.60 

> 

12 

277.6 

0.72 

0.92 

0.8 

0.96 

0.99 

-0.7 

1.02 

0.99 

24 

488.8 

0.51 

0.78 

-0.8 

1.05 

0.99 

-2.3 

1.06 

0.99 

36 

646.1 

0.35 

0.54 

-1.6 

1.08 

0.96 

-1.9 

1.06 

0.94 

48 

701.0 

0.3C 

0.50 

-4.3 

1.2C 

)         0.94 

-1.2 

1.04 

0.91 

60 

819  .  9 

0  . 1? 

I           0.38 

-3.9 

1.1" 

f         0.89 

3.5 

0.90 

0.86 

D  U 

O  J-  -*    •    - 
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4 .   Conclusions  and  Recommendations 

This  study  has  demonstrated  that  the  NOGAPS  model 
performs  exceedingly  well  in  forecasting  five  of  the  six 
parameters  of  the  elliptic  representation  of  a  storm. 
The  maximum  mean  absolute  error  in  forecasting  the  amplitude 
A  is  8.38  (Table  2)  which  is  less  than  1%  of  the  verification 
values  that  range  between  900  and  1,000;  the  maximum 
standard  deviation  of  these  errors  in  6.92.   Similar  positive 
statements  apply  to  the  errors  in  forecasting  e,  R,  X  ,  Y 
as  can  be  seen  from  Tables  1  and  2  and  the  single  high  mode 
close  to  zero  in  the  histograms  (Figures  1-18)  of  absolute 
errors . 

The  autocorrelations  (Table  3)  between  the  errors  in 
successive  forecasts  of  any  of  the  six  parameters  indicate 
that  these  errors  may,  for  all  practical  purposes,  be 
treated  as  independent.   Similarly,  for  each  forecast  period, 
the  errors  in  forecasting  the  parameters  A,  z,    R,  a,  X  ,  Y 
appear  to  be  independent  (Table  4).   What  this  implies  is 
that  a  large  error  in  forecasting  a  parameter  may  not  have 
a  lasting  effect  on  other  forecasts  nor  will  it  have  a 
carry  over  effect  on  forecasting  the  other  parameters. 

Even  at  the  individual  storm  level,  the  mean  forecast 
errors  and  their  standard  deviations  are  quite  small; 
once  again  the  exception  is  the  parameter  a.   Scatter 
plots  of  the  forecasted  values  versus  the  verification 
values  indicated  a  linear  relationship  between  the  two 
sets,  in  several  cases.   Regression  analyses  to  fit  straight 
lines  to  the  data  confirmed  this  observation  (Table  6  and 
Figures  19-30).   When  the  data  was  stratified  according  to 
the  forecasts  period,  e.g.,  all  12  hour  forecasts  are 
treated  as  one  group,  and  a  separate  regression  analysis 
performed  for  each  group  the  lienar  relationship  was 
accentuated  (Table  7). 
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The  overall  conclusion  is  that  the  incorporation  of 
the  SEIS  and  VTP  methodology  within  the  NOGAPS  model  has 
improved  the  storm  tracking  capability  of  NOGAPS  and  the 
elliptic  representation  of  storms  provides  a  good  means 
of  providing  synoptic  level  error  statistics  to  the 
field  forecasters. 

We  propose  the  following  topics  for  further  study 
and  research: 

1.  Determine  the  most  appropriate  probability 
distributions  to  describe  the  probabilistic 
behavior  of  the  forecast  errors.   The  indications 
are  that  the  Weibull  family  may  provide  a  good  fit 
to  the  absolute  error  data. 

2.  Develop  procedures  to  generate  uncertainty  contours/ 
confidence  regions  around  the  forecasted  elliptic 
representations  of  a  storm  based  on  the  probability 
distributions  of  the  forecast  errors. 

3.  Examine  more  data  to  determine  the  functional 
relationship  (if  it  exists)  between  the  forecasted 
and  verification  values  of  the  elliptic  parameters. 
Different  stratification  schemes  for  the  storm 
data  such  as  by  geographic  regions  and  by  climatic 
seasons  could  lead  to  the  identification  of  sources 
of  systematic  errors  and  the  means  of  remediation. 
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